Automated identification of waterpipes on Instagram: an application in feature extraction using Convolutional Neural Network and classification based on Support Vector Machine classifier

1 Background: Instagram (a popular image-based social media app) with millions of posts each day can be used to inform public health interventions and policies but current research relying on image-based data often relies on hand coding of images which is time consuming, costly, and may be subject to researcher bias. What is more, current best practices in automated image classification (e.g., support vector machine (SVM), Backpropagation (BP), and artificial neural network) are limited in their capacity to accurately distinguish between objects within images. Objective: The goal of this study is to demonstrate how convolution neural network (CNN) can be used to extract unique features within an image and how SVM can then be used to classify the image. Methods: Images of waterpipes or hookah (an emerging tobacco product possessing similar harms to that of cigarettes) were collected from Instagram and used in analyses (n=840). CNN was used to extract unique features from images identified to contain waterpipes. SVM classifier was built to distinguish between images with and without waterpipes. Methods for image classification were then compared to show how CNN + SVM classifier could improve accuracy. Results: As the number of the validated training images increased, the total number of extracted features increased. Additionally, as the number of features learned by the SVM classifier increased, the average level of accuracy increased. Overall, 99.5% of the 420 images classified were correctly identified as either hookah or nonhookah images. This level of accuracy was shown to be some improvement over earlier methods that used SVM, CNN or Bag of Features (BOF) alone. Conclusions: CNN extracts more features of the images allowing the SVM classifier to be better